The present invention relates to a device that carries out an interpolation of pixel values for performing magnification, reduction, or other scaling process on an image obtained by a color image pickup apparatus, and particularly relates to an interpolation art suited for performing an interpolation calculation on an image taken by a single-panel color image pickup apparatus, in which a Bayer pattern array is employed as an array of light receiving elements.
Light receiving elements generally have detection sensitivities that are wavelength dependent, and each light receiving element is made to serve as an element for detecting light of a wavelength range for which its sensitivity is highest. Thus in a color image pickup apparatus, for example, a total of three types of light receiving elements that are respectively associated with the three primary colors of R (red), G (green), and B (blue) are respectively positioned in plurality on an image pickup surface. With a single-panel color image pickup apparatus, the light receiving elements for the respective colors must be respectively positioned periodically according to predetermined regular patterns on the same image pickup surface. For example, in the case of a Bayer pattern array, which is used most generally, the respective light receiving elements for the three primary colors of R (red), G (green), and B (blue) are positioned according to the pattern, RGRGRGRGR . . . , along each odd row of a square lattice and according to the pattern, GBGBGBGBGB . . . , along each even row. A honeycomb array, with which the Bayer pattern array is rotated by 45° is also used.
Thus with a single-panel color image pickup apparatus, even if m rows and n columns of light receiving elements are arrayed on the image pickup surface, because each individual light receiving element can only output a pixel value concerning a specific color, a color image constituted of m rows and n columns cannot be obtained from just the pixel values output from the respective light receiving elements of the color image pickup apparatus. For example, because a light receiving element for the R-color outputs only a pixel value for the R color, pixel values for the G and B colors will be missing with the pixel corresponding to the position of this light receiving element. A process of using pixel values, which are positioned in a surrounding region and correspond to the same colors as the missing colors, to interpolate the pixel values of the missing colors is thus performed. Such an interpolation process is generally called “demosaicing.”
The abovementioned “demosaicing” process is normally performed by a linear interpolation, such as determining the average of two or four pixel values in the surroundings of the pixel subject to interpolation. However, with a simple linear interpolation process, a problem occurs in edge enhancement of an image and thus, for example, a “demosaicing” process method, with which the edge enhancement problem is prevented by calculating correction terms and performing further interpolation, is disclosed in Japanese Patent Publication No. 2005-269645A.
Because the abovementioned “demosaicing” process is strictly an interpolation process of interpolating the pixel values of missing colors for each individual pixel, the arrangement of the pixel array (definition of the image) does not change after the process. Thus when magnification, reduction, or other scaling process is performed on an image, an interpolation process for the scaling must be performed anew. This interpolation process is a process of defining new pixels at positions at which light receiving elements do not exist at all on the image pickup surface and determining the pixel values of the new pixels using the pixel values of the existing surrounding pixels. As such an interpolation process for performing scaling, a simple linear interpolation calculation may be performed or a cubic interpolation calculation or other high-order interpolation calculation may be performed.
Thus conventionally, a process was employed in which an interpolation process for demosaicing is performed on so-called raw data output from the respective light receiving elements of a color image pickup apparatus to prepare plane data without missing portions for the respective colors and thereafter, an interpolation process for scaling is executed on the respective color plane data. However, with this conventional method, because a simple liner interpolation calculation is performed as the interpolation process for demosaicing, even if a high-order interpolation calculation, such as a cubic interpolation calculation, is performed as the interpolation process for scaling thereafter, the essential effects of the high-order interpolation calculation cannot be obtained. That is, because a rough interpolation is performed at the demosaicing stage, even if interpolation of high precision is performed at the scaling stage, the proper effects are not obtained. A high-grade magnified image thus cannot be obtained.